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Abstract — Low-density parity-check (LDPC) convolutional 
codes have been shown to exhibit excellent performance un- 
der low-complexity belief-propagation decoding |1|, |2|. This 
phenomenon is now termed threshold saturation via spatial 
coupling. The underlying principle behind this appears to 
be very general and spatially-coupled (SC) codes have been 
successfully applied in numerous areas. Recently, SC regular 
LDPC codes have been proven to achieve capacity universally, 
over the class of binary memoryless symmetric (BMS) channels, 
under belief-propagation decoding (3), |4). 

In 1 5 1, 1 6 1, potential functions are used to prove that the 
BP threshold of SC irregular LDPC ensembles saturates, for 
the binary erasure channel, to the conjectured MAP threshold 
(known as the Maxwell threshold) of the underlying irregular 
ensembles. In this paper, that proof technique is generalized 
to BMS channels, thereby extending some results of (4) to 
irregular LDPC ensembles. We also believe that this approach 
can be expanded to cover a wide class of graphical models 
whose message-passing rules are associated with a Bethe free 
energy. 

Index Terms — convolutional LDPC codes, density evolution, 
entropy functional, potential functions, spatial coupling, thresh- 
old saturation. 

I. Introduction 

Low-density parity-check (LDPC) convolutional codes 
were introduced in Q and shown to have outstanding 
performance under belief -propagation (BP) decoding in 0], 
El, ll8l. The fundamental principle behind this phenomenon 
is described by Kudekar, Richardson, and Urbanke in J3] 
and coined threshold saturation via spatial coupling. For 
the binary erasure channel (BEC), they prove that spatially 
coupling a collection of (d v , d c ) -regular LDPC ensembles 
produces a new (nearly) (d v , d c )-regular ensemble whose 
BP threshold approaches the MAP threshold of the original 
ensemble. Recently, a proof of threshold saturation (to the 
"area threshold") has been given for (d v , d c ) -regular LDPC 
ensembles on binary memoryless symmetric (BMS) channels 
when d v /d c is fixed and d v ,d c are sufficiently large ||4). 
This result implies that SC-LDPC codes achieve capacity 
universally over the class of BMS channels because the "area 
threshold" of regular LDPC codes approaches the Shannon 
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limit uniformly over this class when d v /d c is fixed and d v , d c 
are increased. 

The idea of threshold saturation via spatial coupling has 
recently started a small revolution in coding theory, and 
SC codes have now been observed to universally approach 
the capacity regions of many systems JS], (9), IflOl , ATI . 
Ifl2l . |[T3l , fl4l . 11151 . For spatially-coupled systems with 
suboptimal component decoders, such as message-passing 
decoding of code-division multiple access (CDMA) lfl6ll . ifTTl 
or iterative hard-decision decoding of SC generalized LDPC 
codes [18 1, the threshold saturates instead to an intrinsic 
threshold defined by the suboptimal component decoders. 

SC has also led to new results for i^-SAT, graph coloring, 
and the Curie-Weiss model in statistical physics |fl9l , |20l . 
lETl . For compressive sensing, SC measurement matrices 
were introduced in Il22l . shown to give large improvements 
with Gaussian approximated BP reconstruction in (23], and 
finally proven to achieve the information-theoretic limit 
in [24|. Recent results based on spatial coupling are now 
too numerous to cite thoroughly. 

A different proof technique, based on potential functions, 
is used in ||5] to prove that the BP threshold of spatially- 
coupled (SC) irregular LDPC ensembles saturates to the con- 
jectured MAP threshold (known as the Maxwell threshold) of 
the underlying irregular ensembles. This technique is closely 
related to the analysis in [ 1 9 1 , [20 1 for the Curie- Weiss model, 
the heuristic approach in [25], and the continuum approach 
used to prove threshold saturation for compressed sensing 
in [24|. In this paper, the proof technique based on potential 
functions (in J5], (6]) is extended to BMS channels. This 
extends the results of |0] by proving threshold saturation 
to the Maxwell threshold for BMS channels and a wide 
class of SC irregular LDPC ensembles. The main result is 
summarized in the following theorem whose proof comprises 
the majority of this paper. 

Theorem 1: Consider the SC LDPC(A, p) ensemble de- 
fined in Section IIV-AI and a family of BMS channels c(h) 
that is ordered by degradation and parameterized by entropy, 
h. For any BMS channel c(h) with h less than the potential 
threshold h*, there exists a sufficiently large coupling param- 
eter wq such that, for all w > wq, the SC density evolution 
converges to the perfect decoding solution. 

Many observations, formal proofs, and a large variety 
of applications systems bear evidence to the generality of 



threshold saturation. In particular, the approach taken in this 
paper can be seen as analyzing the average Bethe free energy 
of the SC ensemble in the large system limit [26], ll27l . 
Therefore, it is tempting to conjecture that this approach can 
be applied to more general graphical models by computing 
the average Bethe free energy of the corresponding SC 
system. 

II. Preliminaries 
A. Measure Algebras 

We call a Borel measure x symmetric if 

/ e~ a/2 x{da) = f e- a/2 x(da), 

J-E JE 

for all Borel sets £cM where one of the integrals is finite. 
Let M. = M(R) be the space of finite signed symmetric 
Borel measures on the extended real numbers R. In this work, 
the primary interest is on convex combinations and differ- 
ences of symmetric probability measures that inherit many of 
their properties from A4. Let X C A4 be the convex subset 
of symmetric probability measures. Also, let X<± C M. be the 
subset of differences of symmetric probability measures: 

Xd = {xi -x 2 | Xi,x 2 £ X}. 

In the interest of notational consistency, x is reserved for both 
finite signed symmetric Borel measures and symmetric prob- 
ability measures, and y, z denote differences of symmetric 
probability measures. 

In this space, there are two important binary operators, © 
and S, that denote the variable-node and check-node density 
evolution operations for log-likelihood ratio (LLR) message 
distributions, respectively. The wildcard * is used to represent 
either operator in statements that apply to both operations. 
For example, the shorthand x* n is used to denote 

x * • • • * x . 

" v ' 

TO 

For a polynomial \(a) — YltSo ^nU n with real coeffi- 
cients, we define 

dcg(A) 

A*(x)^ A « x ™ 
n=0 

Now, we give an explicit integral characterization of the 
operators © and S. For xi, x 2 £ M., and any Borel set E C 
1, define 

(xi©x 2 )(£0 = Jx 1 (E-a)x 2 (da), 

(x lffl x 2 )(£)4/ Xl ( 2 tanh- (^§))x 2 (da). 

Associativity, commutativity, and linearity follow. Therefore, 
for measures Xi, x 2 , x 3 £ M and scalars a.\, a 2 £ R, 

xi * (x 2 *x 3 ) = (xi *x 2 ) *x 3 , 
Xl * x 2 = x 2 * Xl, 
(aixi + a 2 x 2 ) * x 3 = ai(xi * x 3 ) + a 2 (x 2 * x 3 ). 



Moreover, the space of symmetric probability measures is 
closed under these binary operations [28]. In a more abstract 
sense, the measure space M along with the multiplication 
operator * forms a commutative unital associative algebra 
and this algebraic structure is induced on the space of 
symmetric probability measures. There is also an intrinsic 
connection between the algebras defined by each operator 
and one consequence is the duality (or conservation) result 
in Proposition [4] For the multiplicative identities in these 
algebras, e® = A and e ffl = Aoo, we define x*° = e* and 
observe the following relationships under the dual operation: 

A ffl x = A Aoo ©x = Aoo. 

In general, however, these operators do not associate 

xi © (x 2 ffl x 3 ) ^ (xi © x 2 ) m x 3 
Xi II (x 2 © x 3 ) ^ (xi ffl x 2 ) © x 3 , 

nor distribute 

xi © (x 2 ffl x 3 ) 7^ (xi © x 2 ) m (xi © x 3 ) 
Xi ffl (x 2 © x 3 ) (xi m x 2 ) © (xi ffl x 3 ). 

B. Partial Ordering by Degradation 

Degradation is an important concept that allows one to 
compare some LLR message distributions. The order im- 
posed by degradation is indicative of relating probability 
measures through a communication channel ||28~I Definition 
4.69]. 

Definition 2: For x g X and / : [0, 1] — > R, define 

J/(x) = y"/(|tanh(f)|)x(rfa). 

For Xi , x 2 £ X, Xi is said to be degraded with respect to x 2 
(denoted xi >z x 2 ), if If(xi) > //(x 2 ) for all concave non- 
increasing /. Furthermore, xi is said to be strictly degraded 
with respect to x 2 (denoted xi >- x 2 ) if xi >^ x 2 and there is 
a concave non-increasing / so that //(xi) > //(x 2 ). 

Degradation defines a partial order, on the space of sym- 
metric probability measures, with maximal element Ao and 
minimal element Aoo- In this paper, the notation for real 
intervals is overloaded and, for example, a half-open interval 
of measures is denoted by 

(xi,x 2 ] 4 { x ' G X | xi -< x' r<x 2 }. 

This partial ordering is also preserved under the binary 
operations as follows. 

Proposition 3 ([28 Lemma 4.80]): Let Xi,x 2 £ X be 
ordered by degradation Xi >z x 2 . For any x 3 £ X, one has 

Xl * x 3 t x 2 * x 3 . 

This ordering is our primary tool in describing relative 
channel quality, and thresholds. For further information see 
Il28l pp. 204-208]. 



C. Entropy Functional for Symmetric Measures 

Entropy is a fundamental quantity in information theory 
and communication, and it is defined as the linear functional, 
H : M — > R, given by the integral 



H(x) 



log 2 (l + e- a )x(c!a). 



The entropy functional is the primary functional used in 
our analysis. It exhibits an order under degradation and, for 
symmetric probability measures Xi y X2, one has 

H(xi) > H(x 2 ). 

The restriction to symmetric probability measures (x G X) 
also implies the bound 

< H(x) < 1. 

The operators © and ffl admit a number of relationships 
under the entropy functional. The following results will prove 
invaluable in the ensuing analysis. Proposition [4] provides 
an important conservation result (also known as the duality 
rule for entropy) and Proposition |5] extends this relation to 
encompass differences of symmetric probability measures. 
Proposition 4 ( H28~\ pp. 196]): For xi,x 2 G X, one has 

H (xi © x 2 ) + H (xi ffl x 2 ) = H (xi) + H (x 2 ) . (1) 

Proposition 5: For Xi, x 2 , X3, X4 G X, one finds that 

H (xi © (x 3 - x 4 )) + H (xi ffl (x 3 - x 4 )) = H (x 3 - x 4 ) , 

H ((xi - x 2 ) © (x 3 - x 4 )) + H ((xi - x 2 ) ffl (x 3 - x 4 )) = 0. 

Proof: Consider the LHS of the first equality, 

H (xi © (x 3 - x 4 )) + H (xi ffl (x 3 - x 4 )) 
= H (xi © x 3 ) + H (xi ffl x 3 ) - H (xi © x 4 ) - H (xi ffl x 4 ) 
= H (xi) + H (x 3 ) - H (xi) - H (x 4 ) (Proposition Si 
= H (x 3 - x 4 ) . 

The second equality follows by expanding the LHS and 
applying the first equality twice. ■ 

Due to the symmetry of the measures the entropy func- 
tional has an equivalent series representation. 

Proposition 6: If x G M, then 



H (x) = x (1) 



(log2)- 1 



2k{2k- 1) 



— f (tanh|) x(da) 



Proof: For a sketch of the proof, see 1129] Lemma 3]. ■ 
Define the linear functional 

Mfc(x) ± J (tanh|) 2fe x(da). 

Since — x 2k is a concave decreasing function over the interval 
[0,1], for symmetric probability measures xi y X2, by 
Definition [2] 



M fe (xi) < M fe (x 2 ). 



Moreover, for x G X, 



It is also easy to see that the functional Mfe(-) takes the 
following product form under the operator ffl, 

M fc (xi fflx 2 ) = M fc (xi)Mfc(x 2 ). 

Also, if yi , y 2 G Xd are the differences of symmetric prob- 
ability measures, then yi(S) = y 2 (M) = (yi * y 2 )(M) = 0. 
This leads to the following result. 

Proposition 7: For yi,y 2 G Xd, the entropy functional 
reduces to 

^ (10S2)- 1 



H(yi) = 

H(yi ffl y 2 ) = 
In particular, 

H (yi ffl yi) = 



M fe (yi), 



4^ 2fc(2fc - 1) 



E 

k=l 



2fc(2fc - 1) 



M k ( yi ) 2 <0. 



with equality iff yi = almost everywhere. 
The above proposition implies an important upper bound on 
the entropy functional for differences of symmetric probabil- 
ity measures under either operator, © or ffl. 

Proposition 8: If xi,x' l7 x 2 ,x 3 G X with x^ y xi, then 

|H ((xi - xi) * (x 2 - x 3 ))| < H (x'j - xi) . 

Proof: We show the result for the operator ffl. The 
extension to © follows from the duality rule for entropy for 
differences of symmetric probability measures, Proposition 
[5] From Proposition [7] 

|H((x' 1 -x 1 )ffl(x 2 -x 3 ))| 



< 



g2l^ L T)' Mfc «- Xl )l^(>< 2 -><3)l 
(log2)- 1 



(«) 



k=l 



E 2fc(2fc- 1) 



M fc (xi-X1)|M*(X2-X8)|, 



00 



< Mfc(x) < 1. 



= H(xi- Xl ), 

where (a) follows from Mk(x'i) < Mfc(xi) and (6) follows 
since < M fc (x a ), M fc (x 3 ) < 1. ■ 
Corollary 9: For xi, x^, x 2 , x 3 , x 4 G X with x^ y x%, one 
has 

|H ((xi - Xl ) * (x 2 - x 3 ) * x 4 )| < H (xi - xi) . 

Proof: This follows from Proposition |8] by replacing x 2 , 
x 3 with x 2 * x 4 and x 3 * x 4 , respectively. ■ 

D. Directional Derivatives 

The main result in this paper is derived using potential 
theory and differential relations. One can avoid the technical 
challenges of differentiation in the abstract space of measures 
by focusing on directional derivatives of functionals that map 
measures to real numbers. 



Definition 10: Let F : M —> K be a functional on M. 
The directional derivative of F at x in the direction y is 



d x F(x)[y]4lim 
d— >o 



F{x + Sy) - F(x) 



whenever the limit exists. 

This definition is naturally extended to higher-order direc- 
tional derivatives using 

d?F(x)[ yi) ...,y„]^d x (•••d x (d x F(x) [ Yl ]) [y 2 ] • • •) [y n ], 
and vectors of measures using, for x = [xi, • • • ,x m ], 

F(x + Sy) - F(x) 
d x F(xyAlim U Z; )=L, 

— <5->0 

whenever the limit exists. Similarly, we can define higher- 
order directional derivatives of functionals on vectors of 
measures. 

The utility of directional derivatives for linear functionals 
is evident from the following Lemma. 

Lemma 11: Let F : A4 — > M. be a linear functional and 
* be an associative, commutative, and linear binary operator. 
Then, for x, y, z e A4, we have 

d x F(x*")[y] =n^(x*("- 1 )*y), 
d 2 x F(x* n )[y 7 z] = n (n - 1) F (x*^ 1 ^ * y * z) . 

Proof: A binary operation * that is associative, commu- 
tative, and linear admits a binomial expansion of the form 



(x+* y r=5> 



x *(n-i) ^y* 



Then, the linearity of F implies that 

F((x + 5y)* n )-F(x*™) 

n 

= 5nF{x< n -^ * y) + ^^(x* ( "~ l) * y")- 

i=2 

Dividing by 5 and taking the limit gives 

d x F(x*")[y] = nFix^^Uy). 

An analogous argument shows the result for the second-order 
directional derivative. ■ 
Remark 12: In general, applying Taylor's theorem to some 
mapping T : X — > X requires advanced mathematical 
machinery. However, in our problem, the entropy functional 
and its interplay with the operators © and EE impose a 
polynomial structure on the functions of interest, obviating 
the need for Frechet derivatives. Therefore, Taylor's theorem 
becomes quite simple for parameterized linear functionals 
4> : [0, 1] -)• R of the form 

<j){t) = F(xi +t(x 2 - xi)) . 



III. Single System 

Let LDPC(A, p) denote the LDPC ensemble with variable- 
node degree distribution A and check-node degree distribution 
p. The edge-perspective degree distributions A, p have an 
equivalent representation in terms of the node-perspective 
degree distributions L, R, namely, 

L'(a) . . R'{a) 



\[a) 



p(a) 



u{iy - ' R'(iy 

This differential relationship is crucial in developing an ap- 
propriate potential functional for the LDPC (A, p) ensemble. 

Density evolution (DE) characterizes the asymptotic per- 
formance of the LDPC(A,/5) ensemble by describing the 
evolution of message distributions with iteration. For this 
ensemble, the DE update is compactly described by 

x^ +1 > =c©A®(p E (xW)), 

where is the variable-node output distribution after i 
iterations of message passing [30|, ll28l . 

We now develop the necessary definitions for the single- 
system potential framework. Included are the potential func- 
tional, fixed points, stationary points, the directional deriva- 
tive of the potential functional, and thresholds. 

Definition 13: Consider a family of BMS channels whose 
LLR distributions c(h) : [0, 1] — >• X are ordered by degra- 
dation and parameterized by their entropy H (c(h)) = h. 
The BP threshold channel of such a family is defined to be 

C BP A c(h B P)> where h BP A 

supjhe [0,1] |x^c(h)©A®( j o E (x))Vxe (Aoo.Ao]}. 

Definition 14: For a family of BMS channels, c(h), the 
MAP threshold is given by h MAP = 

inf (h G [0,1] | liminf ±E[K(X n | F"(c(h)))] > o) . 

Definition 15: The potential functional (or the average 
Bethe free energy), U : X x X -> M, of the LDPC(A,p) 
ensemble is 

%c)^gH(R»(x))H'(l)H(p E 

- L'(1)H (x s p E (x)) - H (c © L® (p s (x))) . 

Remark 16: This potential functional is essentially the 
negative of the replica-symmetric free energies calculated 
in ||29l , ll26ll . iFJTI . When applied to the binary erasure 
channel, it is a constant multiple of the potential function 
defined in [5|. An example of U(x; c) is shown in Fig. [T] 

Definition 1 7: The fixed-point potential, P : X — > R, of 
the LDPC(A, p) ensemble is 

P(x) = %f(x)), 

where f : X -> X satisfies f(x) © A® (p m {x)) = x. 

Remark 18: For regular codes, one can use duality rule for 
entropy to show that the fixed-point potential defined above is 
exactly equal to the negative of the GEXIT integral functional 
denoted by A in ||4] Lemma 26]. A similar relationship with 
EBP GEXIT integrals is believed to be true in general, but 
the authors are not aware of a proof for this. 



(x)) 



o 



o 
z; 
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Definition 23: For a channel c G X, the energy gap is 
defined as 



Fig. 1. Potential functional for (A, p) = (x 2 , a; 5 ) on the binary symmetric 
channel (BSC), with h G {0.40, 0.416, 0.44, 0.469, 0.48}. The x-input is 
chosen to be the binary AWGN channel (BAWGNC) with entropy h'. 

Definition 19: For x G X, 

a) x is a fixed point of density evolution if 

x = c©A® (p E (x)), 

b) x is a stationary point of the potential if, for all y G X&, 

d x [/(x;c)[y] =0. 

Lemma 20: For x, c G X and y G X^, the directional 
derivative of the potential functional with respect to x in the 
direction y is 

d x ?7(x; c)[y] 

= L'(l)H([x-c©A® (p s (x))] © [p' H (x)ffly]). 

Proof: The directional derivative for each of the four 
terms is calculated following the procedure outlined in the 
proof of Lemma QT| The first three terms are 

d x H(i? H (x)) [y]= J R'(l)H(p»(x)ffly), 
d x H(p H (x)) [y]=H(p' H (x)ffly), 
d x H(xfflp H (x)) [y] =H(p i (x)ffly)+H(xHp' i (x)ly) 

( = ) H(p ffl (x)ffly)+H(p' ffl (x)ffly) 
-H(x® [p' s (x) ffly]) , 

where (a) follows from Proposition [5] p' ffl (x) [51 y is the 
difference of probability measures multiplied by the scalar 
p'(l). Since the operators © and EE do not associate, one 
must exercise care in analyzing the last term, 

d x H(c©L® (p H (x)))[y] 

= i'(l)H([c©A®(p ffl (x))] © [p' H (x)ffly]). 

Consolidating the four terms gives the desired result. ■ 
Corollary 21: If x G X is a fixed point of density 
evolution, then it is also a stationary point of the potential 
functional. 

Definition 22: For a channel c G X, define the basin of 
attraction for A^ as 

V(c) = {x G X | x' y c © A®(p H (x')) Vx' G (Aoo.x]} . 



AE(c) 



inf U (x; c) 

xe*\v(c) 



Lemma 24: The following relations are important in char- 
acterizing the thresholds. Suppose hi > hg. Then 

a) c(hi) >- c(h 2 ) 

b) U(x; c(hi)) < f/(x; c(h 2 )), if x ^ A w . 

c) V(c(h x )) C V(c(h a )) =s> X \ V(c(h x )) D X \ V(c(h 2 )) 

d) AB(c(h x )) < AB(c(h 2 )) 

Definition 25: The potential threshold channel of a family, 
c(h), of BMS channels is given by c* = c(h*), where 

h* 4 su P {h G [h BP , 1] | A£(c(h)) > 0} 



Note that, if h < h*, from Lemma |24ji, A£(c(h)) > 0. 

Lemma 26: From (29), ll26l . ||3T1 . the following holds 
for degree distributions without odd-degree checks on any 
BMS channel and any degree distribution on the binary-input 
AWGN channel: 



inf f/(x;c(h)), 



a) liminfiE[H(X"|y n (c(h)))] > 

n— t oo n 

b) h MAP < h*. 

Proof: Since the potential functional is the negative of 
the replica-symmetric free energies in H291 . Il26l . ll3TI . the 
main result of these papers translates directly into the first 
result. For the second result, consider any h > h*. Below, we 
will arrive at a conclusion that h > h MAP , which establishes 
the desired result. Note that from Lemma l24fa. A£(c(h)) < 
0. From the first part of this lemma, 

liminfiE[H (X"|Y"(c(h)))] 

> 



inf [/(x;c(h)) 
xe^\v(c) 



inf U(x; c(h)) > 

= -A(£(c(h))) > 0. 

Hence, h > h MAP . ■ 
Remark 27: This implies that the MAP threshold is upper 
bounded by the potential threshold for the single system. It is 
conjectured that, in general, these two quantities are actually 
equal. Some progress has been made towards proving this 
for special cases ll32l . 

IV. Coupled System 

The potential theory from Section [ill] is now extended to 
spatially-coupled systems. Vectors of measures are denoted 
by underlines (e.g., x) with [x]i = x^. Functionals operating 
on single densities are distinguished from those operating on 
vectors by their input (i.e., J7(x;c) vs. [/(x;c)). Also, for 
vectors x and x', we write x >^ x' if Xj >r x[ for all i, and 
x >- x' if x >^ x' and Xi >- x[ for some i. 

A. Spatial Coupling 

The ideas underlying spatial coupling now appear to be 
quite general. The local coupling in the system allows the 
effect of the perfect information, provided at the boundary, 
to propagate throughout the system. In the large system 
limit, these coupled systems show a significant performance 
improvement. 



The (A, p, N, w) SC ensemble is defined as follows. A 
collection of 2N variable-node groups are placed at all 
positions in J\f v = {1, 2, . . . , 2iV} and a collection of 
2N + (w — 1) check-node groups are placed at all positions 
in j\f c — {1,2,..., 2N + (w — 1)}. For notational conve- 
nience, the rightmost check-node group index is denoted by 
N w = 2N + (w — 1). The integer M is chosen large enough 
so that i) MLi, ML'(l)Rj/R'(l) are natural numbers for 
1 < i < deg(L), 1 < j < deg(R), and ii) ML'(l) is 
divisible by w. 

At each variable-node group, MLi nodes of degree i are 
placed for 1 < i < deg(L). Similarly, at each check-node 
group, ML'{l)Ri/R'(l) nodes of degree i are placed for 
1 < i < deg(i?). At each variable-node and check-node 
group the ML'(l) sockets are partitioned into w equal-sized 
groups using a uniform random permutation. Denote these 
partitions, respectively, by Vf ± j and Vf 2 j at variable-node 
and check-node groups, where 1 < i\ < 2N, 1 < < N w 
and 1 < j < w. The SC system is constructed by connecting 
the sockets from Tfj to Vf + j_ij using uniform random 
permutations. This construction leaves some sockets of the 
check-node groups at the boundaries unconnected and these 
sockets are assigned the binary value (i.e., the socket and 
edge are removed). These values form the seed that gets 
decoding started. 

B. Spatially-Coupled Systems 

Let x] be the variable-node output distribution at node 
i after £ iterations of message passing. Then, the input 
distribution to the i-th check-node group is the normalized 
sum of averaged variable-node output distributions 

-. tu — l 
fc=0 

Backward averaging follows naturally from the setup and is 
essentially the transpose of the forward averaging for the 
check-node output distributions. This model uses uniform 
coupling over a fixed window, but in a more general setting 
window size and coefficient weights could vary from node to 
node. By virtue of the boundary conditions, = Aoo for 
i ^ Af v and all £, and from the relation in (|2}, this implies 
xf ) = Aoo for i £ N c and all 1. 

Generalizing (4] Eqn. 12] to irregular codes gives evolution 
of the variable-node output distributions, 

^"-'•^(sgV^ £'«««-.))• < 3 > 

Making a change of variables, the variable-node output 
distribution evolution in (0 can be rewritten in terms of 
check-node input distributions 

w-l I <w-\ \ 

where q = c when i G N v and q = Aoo otherwise. 
While © is a more natural representation for the underlying 




1 2 i N w N w + l 

Aoo xi x 2 x l0 x Nui Aoo 



Fig. 2. This figure depicts the entropies of xi . ■ ■ ■ , xjy„ in a typical 
iteration. The blue line (solid) corresponds to the two-sided system and the 
red line (dashed) to the one-sided system. The distributions of the one-sided 
system are always degraded with respect to the two-sided system, hence a 
higher entropy. The distributions outside the set {1, ■ ■ ■ , N w } are fixed to 
Aoo for both the systems. 

system, is more mathematically tractable and easily yields 
a coupled potential functional. As such, we adopt the system 
characterized by (@J and refer to it as the two-sided spatially- 
coupled system. 

The two-sided spatially-coupled system is initialized with 

x 2 (0) = A , 1 < i < N w . 

This symmetric initialization, uniform coupling coefficients, 
and symmetric boundary conditions (i.e., seed information) 
induce symmetry on all the message distributions. In partic- 
ular, the two-sided system is fully described by only half the 
distributions because 



for all £. As density evolution progresses, the perfect bound- 
ary information from the left and right sides propagates in- 
ward. This propagation induces a nondecreasing degradation 
ordering on positions I, . . . , \N w /2~\ and a nonincreasing 
ordering on positions \(N W + l)/2] , . . . , N w @ Def. 44]. 

The nondecreasing ordering by degradation introduces a 
degraded maximum at io = N + [-fj, and this maximum 
allows one to define a modified recursion that upper bounds 
the two-sided spatially-coupled system. 

Definition 28 (14]): The one-sided spatially -coupled sys- 
tem is a modification of (j4} defined by fixing the values of 
positions outside M' c = {1, 2, . . . , io}, where io is defined 
as above. As before, the boundary is fixed to Aoo, that is 
= Aqo for i -A/"c an d a U £ More importantly, it also 
fixes the values x$ = f° r *o < i < N w and all £. 

The density evolution update for the one-sided system is 

identical to © for the first io terms, {x^' , xip , . . . , }. But, 

(D 

for the remaining terms, it simply repeats the distribution x s - ; 

in tg\ 
and this implies x 2 - = x,- for ia < i < N w at every step. 

The one-sided and two-sided systems are illustrated in Fig. |2] 

Lemma 29: For the one-sided spatially-coupled system, 

the fixed point resulting from the density evolution satisfies 

Xi^x 4 _!, l<i<N m 
Proof: See E Section IV-D]. ■ 



N w r i 2N I w-l \ 

^;c)=I'(l)E — -H(i? E (x 4 ))+H(^(x ! ))-H(x 4 fflp H (x J )) -£h c©L®(-^p H (x l+J )) (5) 



N w ( w-l I -. iu-1 \ 

dxf/(x;c)[y] = L'(l)^H jq - - £ c,_ fc © A® -$>■(*-**,■) 
i=i V fe=o y 3=o y 



dx%c)[y,z] = 

L'(iy(i)gH 



s=l 



L'(1)/(1)^H 



i ™ * /i 

- V Q _ fc ©A® - Vp ffl (x i _ fe+ 

k=0 \ j=0 

p" H (x,) 



p" B (xQ 

p"(l) 



Yi ffl Zj 



Xj ffl 



p"(l) 



p /a (xj) 




L'(l)A'(l)p'(l) 2 ^ -Hi+(-_-D,^} 



p' H (x 4 ) ffly, 



p' a (x m ) 

p'(i) 



i=l m=max{i-(io-l),l} \ fc=0 



ffl Z r , 



(6) 



(7) 



P '(i) ffl y* 



C. Spatially-Coupled Potential 

Definition 30: The potential functional for a spatially- 
coupled system U : X Nw x X — > K is given by (01. 

Lemma 31: For a spatially-coupled system, the directional 
derivative of the potential functional with respect to x G X Nw 
evaluated in the direction y g A? j *" is given by ([SJ. 

Proof: The proof is similar to the single system case 
and is omitted for brevity. ■ 

Lemma 32: For a spatially-coupled system, the second- 
order directional derivative of the potential functional with 



x X. 



N u 



respect to x evaluated in the direction [y,z] € Af d 
is given by (0. 

Proof: We skip the proof for brevity. ■ 

V. A Proof of Threshold Saturation 

We now prove threshold saturation for the spatially- 
coupled LDPC(A,p) ensemble. Consider a spatially-coupled 
system with potential functional U : X Nw x X — > K as in 
Definition [30] and a parameterization <j> : [0, 1] — > M. 

<f>{t) = U{x + t(x! -x);c), 

where x is a non-trivial fixed point of the one-sided SC 
system resulting from DE. The path endpoint x' is chosen to 
impose a small perturbation on x. For all channels better than 
the potential threshold, c -< c(h*), and any non-trivial fixed 
point, it can be shown that the one-sided SC potential strictly 
decreases along this perturbation. However, since a fixed 
point is also a stationary point of the potential functional, all 
variations in the potential up to the second-order can be made 
arbitrarily small by choosing a large coupling parameter w. 
Thus, one obtains a contradiction to the existence of a non- 
trivial fixed point from the second-order Taylor expansion of 

m 

These ideas are formalized below. A right shift is cho- 
sen for the perturbation. This shift operator is defined in 
Definition [33] and Lemmas [34] [35] discuss its effect on the 



potential functional at this shift and the directional derivative 
along the shift direction [S(x) — xj. Lemma [36l establishes an 
important bound on the second-order directional derivative 
of the potential. Theorem [37] proves threshold saturation. 



Definition 33: The shift operator S 
defined pointwise by 

[SQOh^Aoo, [S(x)]i = xi_i, 

Lemma 34 (j\5j Lemma 4]): Let x € 



X Nm 



X 



2 < i < N w . 
X N ™ be such that 



x; = Xj , for iq < i < N w . Then, the change in the potential 
functional for a spatially-coupled system associated with the 
shift operator is bounded by 

*7(S(x);c)-[/(x;c)<-;7(x l0 ;c). 

Proof: Due to boundary conditions, x; = Xj for io < 
i < N w , the only terms that contribute to U(S(x); c) — C/(x; c) 
are given in (HJ. As the last w values, that is Xi for 2N < 
i < N w , are equal to Xj , we have the desired result. ■ 
Lemma 35: If x y A^ = (Aoo, • • • , Aoo) is a fixed point 
of the one-sided spatially-coupled system, then 

dxE/(x;c)[S(x) -x] =0, 

and x ia ^ V(c) (i.e., x io not in basin of attraction of A^). 
Proof: See Appendix. ■ 
Lemma 36: Let x x £ X Nw be a vector of symmetric prob- 
ability measures, and let x 2 G X N ™ be a vector of coupled 
check-node inputs ordered by degradation, [xa]j h [x2]i-i, 
generated by coupling a vector of variable-node outputs 

x, eX 2N 



^ w—1 



x 2 i = - 

w 



x 2 



k=0 



The second-order directional derivative of U (xj ; c) with 
respect to x x evaluated along [S(x 2 ) — x 2 , S(x 2 ) — x 2 ] can 
be absolutely bounded with 

dl C/(x i; c)[S(x 2 )-x 2 ,S(x 2 )- 



< 



L'(l 



C/(S(x);c) - C/(x;c) = -^yH {R s (x N J) - L'(1)H (p H (x^J) + L'(1)H ( Xjv „ ffl p' ffl ( Xj O) 

/ W — 1 \ / M> — 1 



(8) 




where the constant 

4 7 L'(1) (2/(1) + p'(l) + 2A'(l)p'(l) 2 ) 

is independent of N and «). 

Proof: See Appendix. ■ 
Theorem 37: Consider a family of BMS channels c(h) 
that is ordered by degradation and parameterized by the 
entropy, h. For a spatially-coupled LDPC (A, p) ensemble 
with a coupling window w > K\ iP /(2AE(c(h))), and a 
channel c(h) with h < h*, the only fixed point of density 
evolution is Aoo. 

Proof: Consider a one-sided spatially-coupled system. 
Fix w > Kx,p/(2AE(c(h))). Suppose x >- Aqq is a fixed 
point of density evolution. Let y = S(x) — x and <fr : [0, 1] — > 
R be defined by 

4>{t) = ?7(x + iy;c(h)), 

it is important to note that, for all t £ [0, 1], x + ty = (1 — 
t)x + tS(x) is a vector of probability measures. By linearity 
of the entropy functional and binary operators © and EE], </> is 
a polynomial in t, and thus infinitely differentiable over the 
entire unit interval. The second-order Taylor series expansion 
about t = evaluated at t = 1 provides 

0(0) + 0'(O)(1 - 0) + U"(t )(l - 0) 2 , (9) 



1 



for some to £ [0, 1]. The first and second derivatives of cj> 
are characterized by the first- and second-order directional 
derivatives of U: 

U(x +(t + 5)y; c(h)) - U(x + ty; c(h)) 



4>'{t) 

= d ; 

and similarly, 



lim 

(5->0 



hty f/(x + iy;c(h))[y] 



<t>"(t) =d^ +t yC/(x + ty;c(h))[y ) y]. 
Substituting and rearranging terms in (|9} provides 
5 d I+t y [/ (>< + ioy;c(h))[y,y] 

= f/(S(x);c(h)) - t/(x;c(h)) -dx?7(x;c(h))[S(x) -x] 

= U(S(x);c(h)) - t/(x;c(h)) (LemmaES 

< -C/(x io ;c(h)) (Lemma [34b 

< -AJ5(c(h)). (Lemma [351 and Definition 122b 

Taking an absolute value and applying the second-order 
directional derivative bound from Lemma [36] gives 

— w < 



A£?(c(h)) < 



2w ~ 2AB(c(h))' 

a contradiction. Hence the only fixed point of the one-sided 
system is Aqq. The densities of the one-sided system are 



degraded with respect to the two-sided system, and therefore, 
the only fixed point of the two-sided system is also Aqq . ■ 

VI. Conclusions 

In this paper, the proof technique based on potential 
functions (in 0, |[6l) is extended to BMS channels. This 
extends the results of [4| by proving threshold saturation to 
the Maxwell threshold for BMS channels and the class of 
SC irregular LDPC ensembles. In particular, for any family 
of BMS channels c(h) that is ordered by degradation and 
parameterized by entropy, h, a potential threshold h* is 
defined that satisfies h MAP < h*, where h MAP defines the 
MAP threshold of the underlying LDPC(A, p) ensemble. 

The main result is that, for sufficiently large w, the SC 
DE equation converges to the perfect decoding fixed point 
for any BMS channel c(h) with h < h*. The approach taken 
in this paper can be seen as analyzing the average Bethe free 
energy of the SC ensemble in the large system limit ll26l . 

am 

This result reiterates the generality of the threshold sat- 
uration phenomenon, which is now evident from the many 
observations and proofs that span a wide variety of systems. 
We also believe that this approach can be extended to more 
general graphical models by computing the average Bethe 
free energy of the corresponding SC system. 
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Appendix 

In the appendix, for a vector of measures x i? Xjj is used 
to denote [xjj. 

A. Proof of Lemma \35\ 

Since x is a fixed point of the one-sided spatially coupled 
system, 



w — 1 



in ^-^ 



1 



Ci-fc 



E" ffi 



P @ {Xi-k+j) 



k=0 



3=0 



for 1 < i < iq, and [S(x) — x]j = for io < i < N w . 
The first result follows from applying these relations to the 
directional derivative, given in Lemma [5T| Now, observe that 

^ w — 1 / _^ w — 1 

x *o = ™ E c *o-fc ® A ® - E p B (*o-*+i) 

k=Q V j=0 

1 w — 1 I ^ w — 1 

in * J \ m ^ — ^ 



^c©A® (p ffl (x 40 )) 
Hence x io V(c). 



B. Proof of Lemma \36\ 

Let y = S(x 2 ) — x 2 , with componentwise decomposition 

Yj = [ s (&) - x 2 ]j = x 2 ,i-i - x 2 ,j 

_^ u> — 1 ^ w— 1 

= — / , X2.1-I-A; / , *2,j-k = — (X2,j-M) — x 2,7"); 

k=Q k=0 

where x 2j - = ^00 for j < 1. Referencing Lemma l32l the first 
three terms of the second-order directional derivative are of 
the form 

H (x 3 fflyi fflyj) = — H (x 3 ffl (x 2 ,i -x^i-to) ffl (x 2 ,i-x 2 ,i_i)) , 
w 

by linearity. From Corollary |9] this term is absolutely 
bounded by 

|H(x 3 ffl yi m y 4 )| < — H(x 2 ,j -x 2 ,i_i) . 
10 

The final term is of the form 

|H (x 4 © [x 5 ffl y m ] © [x 6 11 yi ])| 

= |H ([x 4 © (x 5 ll y m )] II [x 6 II yi])| (Proposition |5]l 
= |H (x 6 ffl [x 4 © (x 5 IB y m )] IS y<)| 

= — |H(x 6 m [x 8 -x 7 ] IB [x 2 j -x 2 ,i_i])| 

< — H (x 2 ,i - x 2)i _i) . (Corollary ID 
By telescoping, one observes 



Eh( 



x 2 . 



x 2 ,i_i) = H (x 2 ,jv„ - Aoc) < 1. 



Consolidating the above observations provides the desired 
upper bound. 
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